The present invention relates to an image search system for searching for and retrieving a desired image from a memory device which has a number of images stored thereon.
One of the most important technologies needed across many traditional and emerging applications is the management of visual information. Every day we are bombarded with information presented in the form of images. So important are images in our world of information technology, that we generate literally millions of images every day, and this number keeps escalating with advances in imaging, visualization, video, and computing technologies.
It would be impossible to cope with this explosion of image information unless the images were organized for rapid retrieval on demand. A similar situation occurred in the past for numeric and other structured data, and led to the creation of computerized database management systems. In these systems, large amounts of data are organized into fields, with important or key fields being used to index the databases making search very efficient. These systems, however, are limited by the fact that they work well only with numeric data and short alpha-numeric strings. In the same way, the creation of massive image repositories is of little value unless there are methods for fast and accurate retrieval of desired images from these image databases.
Traditional methods available for searching for and retrieving a desired image have included searching through still images one at a time, or by searching through a list of titles assigned to respective ones of the still images, which titles may or may not be grouped in trees corresponding to subject categories representative of the content of the images. With the method of searching for images one by one, however, problems are encountered in that this method leads to operator eye fatigue and has a low search speed. With the method of search and retrieval from a title list, the images cannot be searched while being observed. Consequently, the accuracy of the search is poor and the search is difficult to perform.
In more modern search and retrieval systems, searching and classifying images is typically based on storing images in a database with descriptive text annotations. The user then searches by inputting a text description of an image and attempting to locate images that have a matching text description. There are numerous disadvantages, however, with using this approach to search or classify images.
One such disadvantage is that a user must decide which visual elements of an image are significant and worth describing. This subjective judgment may overlook various image details that may later be part of image characteristics for which the user is searching. Thus the user may not note or describe specific objects in the background of an image and/or may overlook significant colors, shapes, the presence of persons, or other elements which the user desires in the image. These problems are compounded if the user does not have a clear idea of what is desired. Another disadvantage is that even if the user does have a clear idea of what is desired, the user may have trouble verbalizing this mental image. In order for these prior art searching systems to be effective, the user must describe the desired image in terms that match the stored text description of the image. If the user does not use the precise wording which corresponds to the text description, relevant images will not be retrieved.
Thus, these search and retrieval systems which are based upon image text descriptions may be adequate when the user has a clear idea of the desired image and can verbalize this mental image. However, these systems can be time consuming and frustrating if the user does not have a clear mental picture of the desired image or if the user cannot verbalize a query. In such instances, it should be the task of the search and retrieval system to determine what the user desires, not the responsibility of the user to form his or her thoughts into machine-readable queries. This is a fundamental limitation of textually based search and retrieval systems.
Attempts have been made to overcome the deficiencies of such textually based systems. One example of such an attempt can be found in U.S. Pat. No. 5,802,361 (xe2x80x9cthe ""361 patentxe2x80x9d), which discloses a method and system for searching graphic images and videos, and which is illustrated in FIGS. 9 and 10. The system includes a user interface which allows a user to graphically construct a search inquiry with icons representing image attributes corresponding to image data reflective of the images stored on the system. Referring to FIG. 9, a user inputs a graphical query at 902, specifying a parameter Z and a weight to be accorded to parameter Z. At 904, the system retrieves from an images database 906, and displays to the user, images having the specified parameter Z at or as close as is available to the specified parameter weight. The user, at 908, chooses one or more of the retrieved images as desirable. The system, at 910, recalculates the weight of parameter Z according to the weight of parameter Z of the retrieved images and retrieves from images database 906, and displays to the user, new images having parameter Z at or as close as is available to the recalculated parameter weight. The user may then, at 912, elect to repeat the choosing and recalculating process in another iteration, or to end the search at 914 if desired images have been located.
Although the system embodied in the ""361 patent does obviate some of the disadvantages of textually based searching methods, the system suffers from a number of disadvantages of its own. One of these disadvantages is that although the system does not require a user to enter a textually based query, it does require the user to enter a graphically based query. Thus, this system does nothing to help the situation where a user is having trouble visualizing precisely what is desired in an image. If the user cannot visualize the image sufficiently to enter a text based inquiry, the user will not be able to enter a graphical inquiry. Another disadvantage of the system is that the system can only accord weights to specific parameters entered by the user. Thus, for example if the user specifies parameter Z, the system only iteratively adjusts the weight of parameter Z when a user chooses desirable images. Suppose, for example, a user chooses images heavily weighted toward a second parameter Y, the system cannot recognize this and return images weighted toward parameter Y. Thus, the user would have to recognize a desire for parameter Y without any help from the system (which is often difficult if not impossible), and would have to manually enter a new search query specifying both parameter Z and parameter Y.
It should be noted that the ""361 patent, the disadvantages thereof, and FIGS. 9 and 10 are discussed in more detail below.
What is desired, therefore, is an image search system for searching for and retrieving a desired image from a collection of images which is efficient and allows users to quickly find desired images, which is accurate in returning images likely to be desirable to the user, which does not require the user to verbalize desirable image attributes, which does not require the user to preconceive a mental image of what is desired before the search, which does not require the user to enter a search query, and which is adaptable in that the system readily and automatically adjusts search criteria during the search to reflect a user""s desires.
Accordingly, it is an object of the present invention to provide an image search and retrieval system for searching for and retrieving a desired image from a collection of images which is efficient and allows users to quickly find desired images.
Another object of the present invention is to provide an image search and retrieval system having the above characteristics and which is accurate in returning images likely to be desirable to the user.
A further object of the present invention is to provide an image search and retrieval system having the above characteristics and which does not require the user to verbalize desirable image attributes.
Still another object of the present invention is to provide an image search and retrieval system having the above characteristics and which does not require the user to preconceive a mental image of what is desired before the search.
Yet a further object of the present invention is to provide an image search and retrieval system having the above characteristics and which does not require the user to enter a search query.
Yet another object of the present invention is to provide an image search and retrieval system having the above characteristics and which is adaptable in that the system readily and automatically adjusts search criteria during the search to reflect a user""s desires.
These and other objects of the present invention are achieved by provision of an image search and retrieval system including a computer and a plurality of images accessible by the computer. Each of the images has associated therewith an image profile also accessible by the computer. A plurality of selection probability functions are provided, which are set to initial values at the beginning of the search. Software executing on the computer selects a plurality of images based upon the relationship between the profile for each image and the values of the selection probability functions, and displays the selected images. Software executing on the computer receives an indication of at least one chosen image and adjusts the value of each of the selection probability functions based upon the profiles of the chosen images. The software selects and displays images, receives an indication of at least one chosen image, and adjusts the value of each of the selection probability functions iteratively until the search is terminated.
Preferably, each image profile comprises a plurality of rankings for a plurality of categories, and most preferably, the plurality of rankings comprise a plurality of characteristic functions, each of which represents a probability distribution function which describes the probability of an image being selected given user preferences within a category. Each of the plurality of characteristic functions may take the form of a Gaussian function, or a bell curve, although such is not strictly required.
Also preferably, each of the selection probability functions corresponds to one of the categories, and the initial values of the selection probability functions are set such that statistical weighting among the categories is even and no category is favored. The software executing on the computer preferably selects the images by randomly selecting values for each category, which value is in accordance with the selection probability function for that category, and selects images having characteristic functions corresponding to the randomly selected values for the categories. Preferably, each of the selection probability functions comprises a probability distribution function which describes the probability of a desired image being found, given the initial value of the selection probability function and the profiles of previously chosen images.
The invention and its particular features and advantages will become more apparent from the following detailed description considered with reference to the accompanying drawings.